DAG Nammit

Levels and methods of explainability

Interpretability vs. explainability

• Explainable machine learning: “use of post hoc methods to understand how a model makes predictions”

Useful resources

So what’s the problem with causally interpreting predictive models?

Some causal fallacies in the wild

Interpreting a linear model

Output from an additive logistic regression fit:

              Estimate Std. Error z value Pr(>|z|)
(Intercept)     -0.665      0.134  -4.961    0.000
Sales.calls      0.074      0.060   1.238    0.216
Interactions     0.091      0.056   1.612    0.107
Economy          0.597      0.091   6.589    0.000
Last.upgrade    -0.022      0.005  -4.190    0.000
Discount        -5.950      0.311 -19.106    0.000
Monthly.usage    0.351      0.146   2.406    0.016
Ad.spend         0.602      0.062   9.766    0.000
Bugs.reported    0.259      0.035   7.345    0.000

Correlation matrix

Interpreting an ML model

• No “simple” prediction formula for most black-box models!
• Usually rely on post-hoc techniques to explain
• Most of these require good visualizations
• Readily available in different software (open-source and propprietary)
• Often misapplied and/or misinterpreted!

Retention example (cont.)

Variable importance scores from an XGBoost fit:

Partial dependence (PD) plots

Interpreting the PD plots

• Ad.spend and Discount are important to this (fictional) business because they can be directly manipulated 🎛️

Partial dependence vs. truth! 😱

PD plot (black) vs. true causal relationship (red)

Even the experts slip up!

Statistical Learning with Big Data (fantastic talk!)

So now what?

Directed asyclic graphs (DAGs)

• Useful for representing causal relationships and assumptions
  • Directed: One-sided arrows (→) connect (assumed) causes and effects
  • Asyclic: no directed path can form a closed loop
• Help determine whether the effect(s) of interest can be estimated from available data
• Based on strong assumptions that are often unverifiable

DAGs in machine learning

Assume we have five features (X1–X5) and a response (Y). Causally interpreting an ML model assumes a very particular DAG!

flowchart TB
  X1 --> Y
  X2 --> Y
  X3 --> Y
  X4 --> Y
  X5 --> Y

flowchart TB
  X1 --> X3
  X1 --> Y
  X2 --> X3
  X2 --> Y
  X3 --> X4
  X3 --> Y
  X4 --> Y
  X5 --> Y

Adjustment sets are key

flowchart LR
  X1 --> X3
  X1 --> Y
  X2 --> X3
  X2 --> Y
  X3 --> X4
  X3 --> Y
  X4 --> Y
  X5 --> Y

• Minimal sufficient adjustment set for estimating

  • Total effect of X3 on Y: {X1, X2}
  • Direct effect of X3 on Y: {X1, X2, X4}

• Tools like DAGitty can help automate this!

DAGitty (copy and paste)

dag {
bb="0,0,1,1"
X1 [pos="0.462,0.332"]
X2 [pos="0.425,0.238"]
X3 [exposure,pos="0.532,0.277"]
X4 [pos="0.529,0.396"]
X5 [pos="0.363,0.416"]
Y [outcome,pos="0.439,0.464"]
X1 -> X3
X1 -> Y
X2 -> X3
X2 -> Y
X3 -> X4
X3 -> Y
X4 -> Y
X5 -> Y
}

Useful resources

Retention example (cont.)

Assume strong domain expertise has allowed us to generate the following DAG:

“Causal Interpretations of Black-Box Models”

Mathematical background

The partial dependence (PD) of \(Y\) on \(X_S\) is defined as

\[ \begin{aligned} g_s\left(x_s\right) &= E_{X_c}\left[g\left(x_s, X_c\right)\right] \\ &= \int g\left(x_S, x_C\right) dP\left(x_C\right) \\ &\approx \frac{1}{N}\sum_{i=1}^N g\left(x_S, X_{iC}\right) \end{aligned} \]

Retention example (cont.)

Using another XGBoost model, here’s the estimated PD of Did.renew on Ad.spend, conditional on Monthly.usage and Last.upgrade: … 🥁

Retention example (cont.)

Double/debiased machine learning

Given a causal model, double ML essentially involves three steps:

1. Predict the outcome (\(y\)) from an appropriate adjustment set and get the residuals (\(r_y\))

2. Predict the treatment (\(x\)) from an appropriate adjustment set and get the residuals (\(r_x\))

3. Regress \(r_y\) on \(r_x\) to create a model of the heterogeneous treatment effect

Double ML for Ad.spend

dml_data <- DoubleML::DoubleMLData$new(
  data = ret.trn,                              # training data
  y_col = "Did.renew",                         # response
  d_cols = "Ad.spend",                         # treatment
  x_cols = c("Last.upgrade", "Monthly.usage")  # adjustment set
)
lrnr <- mlr3::lrn("regr.ranger", num.trees = 500)
set.seed(1810)  # for reproducibility
dml_plr = DoubleML::DoubleMLPLR$new(
  dml_data, ml_l = lrnr$clone(), ml_m = lrnr$clone()
)
dml_plr$fit()

# Print results
print(dml_plr)
# ------------------ Fit summary       ------------------
#   Estimates and significance testing of the effect of target variables
#          Estimate. Std. Error t value Pr(>|t|)
# Ad.spend  -0.09634    0.25197  -0.382    0.702

# Compute 95% confidence interval
print(dml_plr$confint())
#               2.5 %   97.5 %
# Ad.spend -0.5901917 0.397511

Challenges with DAG-based inference

• Faithful DAGs seem hard to come by
  • What if you have 800 potential features?
  • Is the right domain expertise even available?
• What about unmeasured confounders?
• DAGs are based on strong (and often unverifiable) assumptions
• Oftentimes multiple reasonable DAGs will exist
  • Sensitivity analysis!

Designed experiments

• RCTs are arguably still the gold standard, but …
  • 😇 There can be ethical concerns
  • 💰 Can be expensive to implement

However…

Ingot cracking example

I’m reminded of an old (but still fantastic) data mining lecture from Richard De Veaux (skip to the 44:30 mark)

• 20,000 lb. ingots made in a giant mold
• Roughtly 25% of ingots develop cracks
• Cracked ingots cost $30,000 to recast
• Roughly 900 observations (ingots) on 149 variables
• What’s causing them to crack?

Ingot cracking example (cont.)

• Lots of iterations, but… “Looks like Chrome(!?)”
• 🕵️ A glass-box model gave clues for generating a hypothesis (i.e., which variable to focus on)
• Follow-up randomized experiments led to substantial improvement!

Adding constraints (where feasible)

• Often useful to constrain the functional form of the model in some way

  • Business considerations
  • Domain knowledge

• Enforcing sparsity (e.g., EBMs with Sparsity)

• Enforcing monotonicty between features and the predicted output can be done in several ways during training (e.g., linear and tree-based models)

  • Can also be accomplished through model editing

Pneumonia example

• Data contains 46 features on 14199 pneumonia patients
  • Patient demographics (e.g., age)
  • 📐 Various measuremnts (e.g., heart rate)
  • 🔬 Lab test results (e.g., WBC)
  • 🩻 Chest x-ray results (e.g., pleural effusion)
• Too many to construct a useful DAG?
• Goal is to predict probability of death (0/1)
• Data from Caruana et al. (2015) and Wang et al. (2022)

Pneumonia example (cont.)

Living past 100 decreases risk?

Pneumonia example (cont.)

Adding monotonic constraints can be helpful!

Pneumonia example (cont.)

Having asthma lowers a patient’s risk of dying from pneumonia?

Pneumonia example (cont.)

According to the doctors, asthmatic patients (A) would likely receive better care earlier (T):

Pneumonia example (cont.)

• If we use the model as is to make hospital admission decisions, asthmatic patients are likely to miss out on care they need
• Interpretability and causal knowledge can help identify such dangerous patterns and improve the model:
  • Force monotonicity (e.g., A > !A)
  • Remove the asthma feature
  • Edit the effect out 😱

GAM Changer

Causal discovery? 🤔

Questions? 🙋

Source: xkcd comic